CLAIMS 

What is claimed is: 

1 . A method comprising: 

describing one or more camera motions that occurred when sequential frames of a 
video were captured, the description for each said frame being provided by: 

a set of displacement curves that describe the one or more camera motions 
in respective horizontal (H), vertical (V), and radial (R) directions; and 

a mean absolute difference (MAD) curve that describes a minimum MAD 
value of the set of displacement curves; and 

detecting a shot boundary in the sequential frames by an abrupt transition of the 
minimum MAD values within the sequential frames, wherein the abrupt transition is 
indicated by a peak in the minimum MAD values described by the MAD curve. 

2. A method as described in claim 1, further comprising characterizing each 
of the described one or more camera motions as one of still, camera shaking, irregular 
camera motion, and one or more regular camera motions. 

3. A method as described in claim 1, wherein the shot boundary is detected 
by examining a central said frame, denoted by k, of a sliding window containing N of said 
frames such that when each of the criteria defined by (a), (b), and (c) as follows are true 
for the minimum MAD value of the central said frame k , the abrupt transition is detected 
at the central said frame k: 

^ MADik) = max(MAD(i)\ i = k-NI 2,..., k + N 12 . 
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MAD(k) > a low MAD sm + fi 

; and 



N/2 , v 

lM4Z)(z) 

wherein a/ ovv% or/,^ and /? are constants, and MAD sm is a second maximum of the TV-frames 
sliding window. 



4. A method as described in claim 1, further comprising generating the set of 
displacement curves by: 

comparing the sequential frames, one to another, utilizing an integral matching 
template to find matching pixels in respective said frames; 

determining displacement of the matching pixels in the compared sequential 
frames; and 

describing the displacement utilizing the set of displacement curves. 

5. A method as described in claim 1, further comprising segmenting the 
video into a plurality of segments based on the detected shot boundary, each said segment 
having one or more said camera motions that are visually consistent. 

6. One or more computer-readable media comprising computer-executable 
instructions that, when executed, perform the method as recited in claim 1. 



7. A method comprising: 

describing one or more camera motions that occurred when sequential frames of a 
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video were captured, the description for each said frame being provided by a set of 
displacement curves that describe the one or more camera motions in respective 
horizontal (H), vertical (V), and radial (R) directions; and 

detecting camera shaking in the video based on the described camera motions in 
the set of displacement curves. 

8. A method as described in claim 7, wherein the detecting includes defining 
motion energy E and variation Var based on the set of displacement curves. 

9. A method as described in claim 7, wherein the detecting includes: 
computing motion energy E and variation Var for a segment of the video from the 

set of displacement curves, wherein the segment includes at least two of the sequential 
frames; 

finding a probabilistic distribution of camera shaking and non-shaking along E 
and Var axes; and 

determining whether the segment includes camera shaking from the probabilistic 
distribution. 

10. A method as described in claim 7, wherein the camera shaking is detected 
for a segment of the video, denoted as t } to k> having at least two of the sequential frames 
by: 

computing motion energy E and variation Var based on the set of displacement 
curves to find a distribution of shaking and non-shaking in E-Var space, wherein E and 
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Var are computed as follows: 



1 2 * i I * V I J ;an , 

Var = 1 sfj^|Vx|^lxe 



wherein " P 7 " is a differential operator to compute relative variation; 

finding a probabilistic distribution of camera shaking and non-shaking along E 
and Var axes by: 

quantifying both the E axis and the Var axis to form a plurality of levels; 

and 

accumulating motion energy E and variation Var in respective said levels 
to form the probabilistic distribution; and 

determining whether the segment includes camera shaking from the probabilistic 
distribution by: 



computing average error probability P E wherein: 




S is a hypothesis that the segment is shaking; 



s is a hypothesis that the segment is non-shaking; 



p ( s ) and H?) 



are prior probabilities for existence of shaking and 



non-shaking, respectively; 



^ is a missed detection area; and 



s is a false detection area; and 
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obtaining optimal decisions T E and TV based on a minimization of P E by a 
likelihood ratio test wherein: 

if the segment satisfies optimal decisions T E and TV, the segment 
includes camera shaking; and 

the optimal decisions T E and TV obtained as follows: 

11. A method as described in claim 7, further comprising generating the set of 
displacement curves by: 

comparing the sequential frames, one to another, utilizing an integral matching 
template to find matching pixels in respective said frames; 

determining displacement of the matching pixels in the compared sequential 
frames; and 

describing the displacement utilizing the set of displacement curves. 

12. One or more computer-readable media comprising computer-executable 
instructions that, when executed, perform the method as recited in claim 7. 

13. A method comprising: 

describing one or more camera motions that occurred when sequential frames of a 
video were captured, the description for each said frame being provided by: 

a set of displacement curves that describe the one or more camera motions 
in respective horizontal (H), vertical (V), and radial (R) directions; 
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a mean absolute difference (MAD) curve that describes a minimum MAD 
value of the set of displacement curves; and 

a MAJ curve that is a qualitative description of the one or more camera 
motions and is determined from the minimum MAD value of the MAD curve; and 
characterizing each of the described one or more camera motions as one of still, 
camera shaking, irregular camera motion, and one or more regular camera motions. 

14. A method as described in claim 13, wherein the qualitative description of 
camera motion includes still, horizontal, vertical and radial. 

15. A method as described in claim 13, further comprising generating the set 
of displacement curves by: 

comparing the sequential frames, one to another, utilizing an integral matching 
template to find matching pixels in respective said frames; 

determining displacement of the matching pixels in the compared sequential 
frames; and 

describing the displacement utilizing the set of displacement curves. 

16. A method as described in claim 13, wherein the characterizing includes: 
morphological filtering of the set of displacement curves by: 

performing an opening operation to remove unstable motions from the set 
of displacement curves to form a set of regular motion displacement curves; and 
performing a closing operation on the set of regular motion displacement 
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curves to connect interrupted regular motions to form a set of connected regular 
motion displacement curves; 

detecting shot boundaries based on the set of connected regular motion 
displacement curves; and 

merging overlap segments of the set of connected regular motion displacement 
curves to form one or more motion segments which describe regular camera motions that 
are included in at least two of the set of connected regular motion displacement curves. 

17. A method as described in claim 13, wherein the characterizing further 
comprises: 

defining motion energy E and variation V based on the set of displacement curves 
to detect the camera shaking and the irregular camera motion; and 
detecting the still camera motion from the MAJ curve. 

18. A method as described in claim 13, wherein the one or more regular 
camera motions are selected from the group consisting of: 

panning; 

tilting; 

zooming; 

tracking; 

booming; 

dollying; and 

any combination thereof. 
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19. One or more computer-readable media comprising computer-executable 
instructions that, when executed, perform the method as recited in claim 13. 

20. A method comprising: 

analyzing video having sequential frames to determine one or more camera 
motions that occurred when sequential frames of the video were captured; and 
describing the one or more camera motions for each said frame by: 

a set of displacement curves that describe the one or more camera motions 
in respective horizontal (H), vertical (V), and radial (R) directions; 

a mean absolute difference (MAD) curve that relates a minimum MAD 
value from the set of displacement curves; and 
a major motion (MAJ) curve that: 

is generated from the minimum MAD value; and 
provides one or more qualitative descriptions that describe the one 
or more camera motions as one of still, vertical, horizontal and radial. 

21. A method as described in claim 20, further comprising detecting a shot 
boundary in the sequential frames from an abrupt transition of respective said minimum 
MAD values. 

22. A method as described in claim 20, further comprising detecting a shot 
boundary in the sequential frames from an abrupt transition by examining a central said 
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frame, denoted by k, of a sliding window containing N of said frames such that when 
each of the criteria defined by (a), (b), and (c) as follows are true for the minimum MAD 
value of the central said frame k , the abrupt transition is detected at the central said frame 

k: 

(a) MAD(k) = max(MAD(i)), i = k-NI 2,..., k + N/2. 

(b) MAD(k) ~ a 'o» MAD s«, + P 

: and 

Nil , . 

ZMAD(i) 

MAD(k)>a high ^ + fi 

9 

wherein cci 0Wy ahigh and p are constants, and MAD sm is a second maximum of the TV-frames 
sliding window. 



23. A method as described in claim 20, further comprising detecting camera 
shaking in the video by defining motion energy E and variation Var based on the set of 
displacement curves. 



24. A method as described in claim 20, further comprising detecting camera 
shaking by: 

computing motion energy E and variation Var for a segment of the video from the 
set of displacement curves, wherein the segment includes at least two of the sequential 
frames; 

finding a probabilistic distribution of camera shaking and non-shaking along E 
and Var axes; and 
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determining whether the segment includes camera shaking from the probabilistic 
distribution. 



25. A method as described in claim 20, further comprising detecting camera 
shaking for a segment of the video, denoted as tj to t 2 , having at least two of the 
sequential frames by: 

computing motion energy E and variation Var based on the set of displacement 
curves to find a distribution of shaking and non-shaking in E-Var space, wherein E and 
Var are computed as follows: 



wherein " V" is a differential operator to compute relative variation; 

finding a probabilistic distribution of camera shaking and non-shaking along E 
and Var axes by: 

quantifying both the E axis and the Var axis to form a plurality of levels; 

and 

accumulating motion energy E and variation Var in respective said levels 
to form the probabilistic distribution; and 

determining whether the segment includes camera shaking from the probabilistic 
distribution by: 





computing average error probability Pe wherein: 
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p E =p is)l p(x\S)dx+p{s)[ p{x\s)dx 



S is a hypothesis that the segment is shaking; 



s is a hypothesis that the segment is non-shaking; 

and Ks) are prior probabilities for existence of shaking and 



non-shaking, respectively; 

R s is a missed detection area; and 

Rs is a false detection area; and 
obtaining optimal decisions T E and 7> based on a minimization of P E by a 
likelihood ratio test wherein: 

if the segment satisfies optimal decisions T E and 7>, the segment 
includes camera shaking; and 

the optimal decisions T E and 7> obtained as follows: 



26. A method as described in claim 20, further comprising characterizing each 
of the described one or more camera motions as one of still camera motion, camera 
shaking, irregular camera motion, and one or more regular camera motions. 

27. A method as described in claim 26, wherein the one or more regular 
camera motions are selected from the group consisting of: 

panning; 
tilting; 



P(S)P(E\S) J 

p(s)p(£\s) > 
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zooming; 

tracking; 

booming; 

dollying; and 

a combination thereof. 

28. A method as described in claim 20, further comprising characterizing each 
of the described one or more camera motions as one or more regular camera motions by: 

morphological filtering of the set of displacement curves by: 

performing an opening operation to remove unstable motions from the set 

of displacement curves to form a set of regular motion displacement curves; and 
performing a closing operation on the set of regular motion displacement 

curves to connect interrupted regular motions to form a set of connected regular 

motion displacement curves; 

detecting shot boundaries based on the set of connected regular motion 
displacement curves; and 

merging overlap segments of the set of connected regular motion displacement 
curves to form one or more motion segments which describe regular camera motions that 
are included in at least two of the set of connected regular motion displacement curves. 

29. A method as described in claim 20, wherein the analyzing is performed 

by: 
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executing a Mean Absolute Difference (MAD) algorithm based on luminance (L), 
wherein: 

luminance (L) is utilized to match pixels in respective first and second said 
frames; and 

the MAD between the first said frame, denoted as k, and the second said 
frame, denoted as £+7, is computed as follows: 

MAD(Ax)=— ^\L(x + Ax 9 k + l)-L(x 9 k) 
N xeT 

wherein J is a set of the matching pixels and "AT is a total number of the 
matching pixels; and 

estimating camera motion through error minimization in parameter state space (H, 
V, R\ which respectively denote the horizontal, the vertical, and the radial directions, 
wherein: 

for each said direction (H, V, R), optimal matching is obtained when a 
minimum MAD is reached as follows: 

(H 9 V,R) = argminMAD(Ax) 

Axeiff 

iPis a state space of a plurality of camera motion candidates; and 
the camera motion candidate that yields a best match is an optimal 
estimation of camera motion. 



30. A method as described in claim 20, further comprising generating the set 
of displacement curves by: 

comparing the sequential frames, one to another, utilizing an integral matching 
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template to find matching pixels in respective said frames; 

determining displacement of the matching pixels in the compared sequential 
frames; and 

describing the displacement utilizing the set of displacement curves. 

31. One or more computer-readable media comprising computer-executable 
instructions that, when executed, perform the method as recited in claim 20. 

32. A computer comprising: 
video having sequential frames; 

a set of displacement curves that describe one or more camera motions that 
occurred when sequential frames of a video were captured in respective horizontal (H), 
vertical (V), and radial (R) directions; 

a mean absolute difference (MAD) curve that relates a minimum MAD value of 
the set of displacement curves for each said frame; and 

a major motion (MAJ) curve that is a qualitative description of camera motion for 
each said frame and is determined from the minimum MAD value of the MAD curve; 
and 

a video analysis module this is executable to: 

detect a shot boundary in the sequential frames from an abrupt transition 
of respective said minimum MAD values; 

detect camera shaking in the sequential frames based on the described 
camera motions in the set of displacement curves; and 
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characterize each of the one or more camera motions as one of still camera 
motion, camera shaking, irregular camera motion, and one or more regular camera 
motions. 



33. A computer as described in claim 32, wherein the video analysis module is 
executable to generate the set of displacement curves by: 

comparing the sequential frames, one to another, utilizing an integral matching 
template to find matching pixels in respective said frames; 

determining displacement of the matching pixels in the compared sequential 
frames; and 

describing the displacement utilizing the set of displacement curves. 

34. A computer as described in claim 32, wherein the video; analysis module is 
executable to: 

executing a Mean Absolute Difference (MAD) algorithm based on luminance (L), 
wherein: 

luminance (L) is utilized to match pixels in respective first and second said 
frames; and 

the MAD between the first said frame, denoted as k 9 and the second said 
frame, denoted as £+7, is computed as follows: 

MAD(Ax)=— £ |l(jc + Ajc, k + l)-L(x 9 k] 
N xeT 

wherein T is a set of the matching pixels and "AT is a total number of the 
matching pixels; and 
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estimating camera motion through error minimization in parameter state space (//, 
V 9 R), which respectively denote the horizontal, the vertical, and the radial directions, 
wherein: 

for each said direction (//, V, R), optimal matching is obtained when a 

minimum MAD is reached as follows: 

(H,V,R) = argminMAD(Ax) 
IFis a state space of a plurality of camera motion candidates; and 
the camera motion candidate that yields a best match is an optimal 

estimation of camera motion. 

35. A computer as described in claim 32, wherein the video analysis module is 
executable to detect the shot boundary by examining a central said frame, denoted by k, 
of a sliding window containing N of said frames such that when each of the criteria 
defined by (a), (b), and (c) as follows are true for the minimum MAD value of the central 
said frame k , the abrupt transition is detected at the central said frame k: 

^ MAD(k) = max(MAD(i)) 9 i=k-N/ 2,..., k + N 1 2 . 



MAD(k) > a low MAD sm + fi 

; and 

N/2 g v 

(c) MAD(k)>a high ±^ + p 



wherein ai OWy ochigh and /?are constants, and MAD sm is a second maximum of the AT-frames 
sliding window. 



36. A computer as described in claim 32, wherein the video analysis module is 
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executable to detect camera shaking by defining motion energy E and variation Var based 
on the set of displacement curves. 

37. A computer as described in claim 32, wherein the video analysis module is 
executable to detect camera shaking by: 

computing motion energy E and variation Var for a segment of the video from the 
set of displacement curves, wherein the segment includes at least two of the sequential 
frames; 

finding a probabilistic distribution of camera shaking and non-shaking along E 
and Var axes; and 

determining whether the segment includes camera shaking from the probabilistic 
distribution. 

38. A computer as described in claim 32, wherein the video analysis module is 
executable to detect camera shaking for a segment of the video, denoted as tj to t 2 , having 
at least two of the sequential frames by: 

computing motion energy E and variation Var based on and the set of 
displacement curves to find a distribution of shaking and non-shaking in E-Var space, 
wherein E and Var are computed as follows: 





wherein " V" is a differential operator to compute relative variation; 
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finding a probabilistic distribution of camera shaking and non-shaking along E 
and Var axes by: 

quantifying both the E axis and the Var axis to form a plurality of levels; 

and 

accumulating motion energy E and variation Var in respective said levels 
to form the probabilistic distribution; and 

determining whether the segment includes camera shaking from the probabilistic 
distribution by: 

computing average error probability P E wherein: 

P E =P(S)1 p(x\S)dx+p(s)l p(x\s)dx 
S is a hypothesis that the segment is shaking; 
5 is a hypothesis that the segment is non-shaking; 

p ( s ) and are prior probabilities for existence of shaking and 
non-shaking, respectively; 
R- 

5 is a missed detection area; and 

D 

5 is a false detection area; and 
obtaining optimal decisions T E and 7> based on a minimization of P E by a 
likelihood ratio test wherein: 

if the segment satisfies optimal decisions T E and 7>, the segment 
includes camera shaking; and 

the optimal decisions T E and 7> obtained as follows: 
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39. A computer as described in claim 32, wherein the video analysis module is 
executable to characterize by: 

morphological filtering of the set of displacement curves by: 

performing an opening operation to remove unstable motions from the set 

of displacement curves to form a set of regular motion displacement curves; and 
performing a closing operation on the set of regular motion displacement 

curves to connect interrupted regular motions to form a set of connected regular 

motion displacement curves; 

detecting shot boundaries based on the set of connected regular motion 
displacement curves; and 

merging overlap segments of the set of connected regular motion displacement 
curves to form one or more motion segments which describe regular camera motions that 
are included in at least two of the set of connected regular motion displacement curves. 

40. A computer as described in claim 32, wherein the regular camera motions 
are selected from the group consisting of: 

panning; 

tilting; 

zooming; 

tracking; 

booming; 
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dollying; and 

a combination thereof. 

41 . A computer as described in claim 32, wherein the video analysis module is 
executable to characterize by: 

defining motion energy E and variation Abased on the set of displacement curves 
to detect the camera shaking and the irregular camera motion; and 

detecting the still camera motion from the MAJ curve. 
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